Mitral Annular Disjunction: An Under-Recognized Entity in Pediatrics.

Mitral annular disjunction (MAD) is a rare and under-recognized entity in the pediatric population. We present 2 cases of MAD in previously healthy pediatric patients and highlight clinical scenarios where MAD should be suspected.

Underrecognition and Suboptimal Quality of Care for Nonalcoholic Fatty Liver Disease Cirrhosis in Primary Care Patients with Diabetes Mellitus.

BACKGROUND: Nonalcoholic fatty liver disease (NAFLD) is a leading cause of cirrhosis but is underrecognized in primary care. Cirrhosis management requires complex monitoring, and the quality of care (QoC) for NAFLD cirrhosis patients in primary care may be inadequate. METHODS: In this retrospective-prospective cohort study of primary care patients with diabetes mellitus, we identified patients with NAFLD cirrhosis by 1) evidence of cirrhosis from abdominal imaging identified by natural language processing, or 2) existence of International Classification of Diseases code for cirrhosis. A finding of either was followed by manual chart review for confirmation of both cirrhosis and NAFLD. We then determined if cirrhosis care measures were up-to-date, including hepatitis A and B vaccination, Model for End-Stage Liver Disease score components, esophagogastroduodenoscopy, and hepatocellular carcinoma screening. We created a composite score quantifying overall QoC (scale 0-8), with high QoC defined as ≥6 points. RESULTS: Among 3,028 primary care patients with diabetes mellitus, we identified 51 (1.7%) with NAFLD cirrhosis. Although 78% had ≥3 average primary care visits/year, only 24% completed hepatocellular carcinoma screening at least annually in at least 75% of years since diagnosis. The average QoC composite score was 4.9 (SD 2.4), and less than one-third had high QoC. CONCLUSIONS: NAFLD cirrhosis is prevalent but underdiagnosed in primary care, and receipt of comprehensive QoC was suboptimal. Given the rising incidence of NAFLD cirrhosis, primary care providers need improved awareness and mechanisms to ensure high QoC for this population.

Quantitative Assessment of Myocardial Ischemia With Positron Emission Tomography.

Recent advances in positron emission tomography (PET) technology and reconstruction techniques have now made quantitative assessment using cardiac PET readily available in most cardiac PET imaging centers. Multiple PET myocardial perfusion imaging (MPI) radiopharmaceuticals are available for quantitative examination of myocardial ischemia, with each having distinct convenience and accuracy profile. Important properties of these radiopharmaceuticals ( 15 O-water, 13 N-ammonia, 82 Rb, 11 C-acetate, and 18 F-flurpiridaz) including radionuclide half-life, mean positron range in tissue, and the relationship between kinetic parameters and myocardial blood flow (MBF) are presented. Absolute quantification of MBF requires PET MPI to be performed with protocols that allow the generation of dynamic multiframes of reconstructed data. Using a tissue compartment model, the rate constant that governs the rate of PET MPI radiopharmaceutical extraction from the blood plasma to myocardial tissue is calculated. Then, this rate constant ( K1 ) is converted to MBF using an established extraction formula for each radiopharmaceutical. As most of the modern PET scanners acquire the data only in list mode, techniques of processing the list-mode data into dynamic multiframes are also reviewed. Finally, the impact of modern PET technologies such as PET/CT, PET/MR, total-body PET, machine learning/deep learning on comprehensive and quantitative assessment of myocardial ischemia is briefly described in this review.

Application of a Domain-specific BERT for Detection of Speech Recognition Errors in Radiology Reports.

Purpose: To develop radiology domain-specific bidirectional encoder representations from transformers (BERT) models that can identify speech recognition (SR) errors and suggest corrections in radiology reports. Materials and Methods: A pretrained BERT model, Clinical BioBERT, was further pretrained on a corpus of 114 008 radiology reports between April 2016 and August 2019 that were retrospectively collected from two hospitals. Next, the model was fine-tuned on a training dataset of generated insertion, deletion, and substitution errors, creating Radiology BERT. This model was retrospectively evaluated on an independent dataset of radiology reports with generated errors (n = 18 885) and on unaltered report sentences (n = 2000) and prospectively evaluated on true clinical SR errors (n = 92). Correction Radiology BERT was separately trained to suggest corrections for detected deletion and substitution errors. Area under the receiver operating characteristic curve (AUC) and bootstrapped 95% CIs were calculated for each evaluation dataset. Results: Radiology-specific BERT had AUC values of >.99 (95% CI: >0.99, >0.99), 0.94 (95% CI: 0.93, 0.94), 0.98 (95% CI: 0.98, 0.98), and 0.97 (95% CI: 0.97, 0.97) for detecting insertion, deletion, substitution, and all errors, respectively, on the independently generated test set. Testing on unaltered report impressions revealed a sensitivity of 82% (28 of 34; 95% CI: 70%, 93%) and specificity of 88% (1521 of 1728; 95% CI: 87%, 90%). Testing on prospective SR errors showed an accuracy of 75% (69 of 92; 95% CI: 65%, 83%). Finally, the correct word was the top suggestion for 45.6% (475 of 1041; 95% CI: 42.5%, 49.3%) of errors. Conclusion: Radiology-specific BERT models fine-tuned on generated errors were able to identify SR errors in radiology reports and suggest corrections.Keywords: Computer Applications, Technology Assessment Supplemental material is available for this article. © RSNA, 2022See also the commentary by Abajian and Cheung in this issue.

Prediction of future healthcare expenses of patients from chest radiographs using deep learning: a pilot study.

Our objective was to develop deep learning models with chest radiograph data to predict healthcare costs and classify top-50% spenders. 21,872 frontal chest radiographs were retrospectively collected from 19,524 patients with at least 1-year spending data. Among the patients, 11,003 patients had 3 years of cost data, and 1678 patients had 5 years of cost data. Model performances were measured with area under the receiver operating characteristic curve (ROC-AUC) for classification of top-50% spenders and Spearman ρ for prediction of healthcare cost. The best model predicting 1-year (N = 21,872) expenditure achieved ROC-AUC of 0.806 [95% CI 0.793-0.819] for top-50% spender classification and ρ of 0.561 [0.536-0.586] for regression. Similarly, for predicting 3-year (N = 12,395) expenditure, ROC-AUC of 0.771 [0.750-0.794] and ρ of 0.524 [0.489-0.559]; for predicting 5-year (N = 1779) expenditure ROC-AUC of 0.729 [0.667-0.729] and ρ of 0.424 [0.324-0.529]. Our deep learning model demonstrated the feasibility of predicting health care expenditure as well as classifying top 50% healthcare spenders at 1, 3, and 5 year(s), implying the feasibility of combining deep learning with information-rich imaging data to uncover hidden associations that may allude to physicians. Such a model can be a starting point of making an accurate budget in reimbursement models in healthcare industries.

Clinical language search algorithm from free-text: facilitating appropriate imaging.

BACKGROUND: The comprehensiveness and maintenance of the American College of Radiology (ACR) Appropriateness Criteria (AC) makes it a unique resource for evidence-based clinical imaging decision support, but it is underutilized by clinicians. To facilitate the use of imaging recommendations, we develop a natural language processing (NLP) search algorithm that automatically matches clinical indications that physicians write into imaging orders to appropriate AC imaging recommendations. METHODS: We apply a hybrid model of semantic similarity from a sent2vec model trained on 223 million scientific sentences, combined with term frequency inverse document frequency features. AC documents are ranked based on their embeddings' cosine distance to query. For model testing, we compiled a dataset of simulated simple and complex indications for each AC document (n = 410) and another with clinical indications from randomly sampled radiology reports (n = 100). We compare our algorithm to a custom google search engine. RESULTS: On the simulated indications, our algorithm ranked ground truth documents as top 3 for 98% of simple queries and 85% of complex queries. Similarly, on the randomly sampled radiology report dataset, the algorithm ranked 86% of indications with a single match as top 3. Vague and distracting phrases present in the free-text indications were main sources of errors. Our algorithm provides more relevant results than a custom Google search engine, especially for complex queries. CONCLUSIONS: We have developed and evaluated an NLP algorithm that matches clinical indications to appropriate AC guidelines. This approach can be integrated into imaging ordering systems for automated access to guidelines.

AI MSK clinical applications: cartilage and osteoarthritis.

The advancements of artificial intelligence (AI) for osteoarthritis (OA) applications have been rapid in recent years, particularly innovations of deep learning for image classification, lesion detection, cartilage segmentation, and prediction modeling of future knee OA development. This review article focuses on AI applications in OA research, first describing machine learning (ML) techniques and workflow, followed by how these algorithms are used for OA classification tasks through imaging and non-imaging-based ML models. Deep learning applications for OA research, including analysis of both radiographs for automatic detection of OA severity, and MR images for detection of cartilage/meniscus lesions and cartilage segmentation for automatic T2 quantification will be described. In addition, information on ML models that identify individuals at high risk of OA development will be provided. The future vision of machine learning applications in imaging of OA and cartilage hinges on implementation of AI for optimizing imaging protocols, quantitative assessment of cartilage, and automated analysis of disease burden yielding a faster and more efficient workflow for a radiologist with a higher level of reproducibility and precision. It may also provide risk assessment tools for individual patients, which is an integral part of precision medicine.

Algorithmic Prediction of Delayed Radiology Turn-Around-Time during Non-Business Hours.

RATIONALE AND OBJECTIVES: Radiology turnaround time is an important quality measure that can impact hospital workflow and patient outcomes. We aimed to develop a machine learning model to predict delayed turnaround time during non-business hours and identify factors that contribute to this delay. MATERIALS AND METHODS: This retrospective study consisted of 15,117 CT cases from May 2018 to May 2019 during non-business hours at two hospital campuses after applying exclusion criteria. Of these 15,177 cases, 7,532 were inpatient cases and 7,585 were emergency cases. Order time, scan time, first communication by radiologist, free-text indications, and other clinical metadata were extracted. A combined XGBoost classifier and Random Forest natural language processing model was trained with 85% of the data and tested with 15% of the data. The model predicted two measures of delay: when the exam was ordered to first communication (total time) and when the scan was completed to first communication (interpretation time). The model was analyzed with the area under the curve (AUC) of receiver operating characteristic (ROC) and feature importance. Source code: https://bit.ly/2UrLiVJ RESULTS: The algorithm reached an AUC of 0.85, with a 95% confidence interval [0.83, 0.87], when predicting delays greater than 245 minutes for "total time" and 0.71, with a 95% confidence interval [0.68, 0.73], when predicting delays greater than 57 minutes for "interpretation time". At our institution, CT scan description (e.g. "CTA chest pulmonary embolism protocol"), time of day, and year in training were more predictive features compared to body part, inpatient status, and hospital campus for both interpretation and total time delay. CONCLUSION: This algorithm can be applied clinically when a physician is ordering the scan to reasonably predict delayed turnaround time. Such a model can be leveraged to identify factors associated with delays and emphasize areas for improvement to patient outcomes.

Longitudinal Analysis of the Effect of Repeated Transarterial Chemoembolization for Liver Cancer on Portal Venous Pressure.

OBJECTIVES: Investigate long-term effects of repeated transarterial chemoembolization (TACE) on portal venous pressure (PVP) using non-invasive surrogate markers of portal hypertension. METHODS: Retrospective, Institutional Review Board-approved study. 99 patients [hepatocellular carcinoma (HCC) group (n=57); liver metastasis group (n=42)] who underwent 279TACEs and had longitudinal pre-/post-therapy contrast-enhanced-MRI (n=388) and complete blood work were included. Outcomes of interest were platelet count (PC), spleen volume, ascites and portosystemic collaterals. Variables included TACE type/number, tumor type, microcatheter location, Child-Pugh, baseline tumor burden (tumor number/total/largest size), vessel invasion, alpha-fetoprotein, Eastern Cooperative Oncology Group (ECOG) performance status, and Model for End-Stage Liver Disease (MELD) score. Generalized Estimating Equations assessed the associations between TACE and outcomes. Power analysis determined the sample size was sufficient. RESULTS: No significant change in PC over time was observed in either groups, regardless of liver function (P>0.05). Baseline spleen volume was 226 cm3 for metastatic group, and was larger by 204 cm3 for HCC group (P<0.001). Spleen volume increased by 20 cm3 (95%CI: 8-32; P=0.001) for both groups after 1stTACE and by 16cm3/TACE (P=0.099) over the full follow-up (up to 9TACEs). Spleen volume also tended to increase by 23cm3 (95%CI: -1-48; P=0.064) with higher tumor burden. Odds of developing moderate/severe ascites for metastatic patients was decreased by 0.5 (95%CI: 0.3-0.9; P=0.014), regardless of the Child-Pugh, and increased by 1.5 (95%CI: 1.2-1.9; P<0.001) among HCC patients with unstable Child-Pugh, whereas no change was noted with stable Child-Pugh. HCC patients with unstable Child-Pugh demonstrated a significant increase in portosystemic collaterals number over time (P=0.008). PVP-related complications such as variceal bleeding post-TACE were low (0.4%). CONCLUSION: Repeated TACEs did seem to have an impact on PVP. However, the increase in PVP had marginal effects with low portal hypertension-related complications.

Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing.

BACKGROUND: A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text. METHODS: We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n = 4997 for spine MRI; n = 839 for head MRI), validation (n = 1071 for spine MRI, fivefold cross-validation used for head MRI), and test (n = 1071 for spine MRI; n = 151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku. RESULTS: The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment. CONCLUSION: We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.

CT-less Direct Correction of Attenuation and Scatter in the Image Space Using Deep Learning for Whole-Body FDG PET: Potential Benefits and Pitfalls.

PURPOSE: To demonstrate the feasibility of CT-less attenuation and scatter correction (ASC) in the image space using deep learning for whole-body PET, with a focus on the potential benefits and pitfalls. MATERIALS AND METHODS: In this retrospective study, 110 whole-body fluorodeoxyglucose (FDG) PET/CT studies acquired in 107 patients (mean age ± standard deviation, 58 years ± 18; age range, 11-92 years; 72 females) from February 2016 through January 2018 were randomly collected. A total of 37.3% (41 of 110) of the studies showed metastases, with diverse FDG PET findings throughout the whole body. A U-Net-based network was developed for directly transforming noncorrected PET (PETNC) into attenuation- and scatter-corrected PET (PETASC). Deep learning-corrected PET (PETDL) images were quantitatively evaluated by using the standardized uptake value (SUV) of the normalized root mean square error, the peak signal-to-noise ratio, and the structural similarity index, in addition to a joint histogram for statistical analysis. Qualitative reviews by radiologists revealed the potential benefits and pitfalls of this correction method. RESULTS: The normalized root mean square error (0.21 ± 0.05 [mean SUV ± standard deviation]), mean peak signal-to-noise ratio (36.3 ± 3.0), mean structural similarity index (0.98 ± 0.01), and voxelwise correlation (97.62%) of PETDL demonstrated quantitatively high similarity with PETASC. Radiologist reviews revealed the overall quality of PETDL. The potential benefits of PETDL include a radiation dose reduction on follow-up scans and artifact removal in the regions with attenuation correction- and scatter correction-based artifacts. The pitfalls involve potential false-negative results due to blurring or missing lesions or false-positive results due to pseudo-low-uptake patterns. CONCLUSION: Deep learning-based direct ASC at whole-body PET is feasible and potentially can be used to overcome the current limitations of CT-based approaches, benefiting patients who are sensitive to radiation from CT.Supplemental material is available for this article.© RSNA, 2020.

High precision localization of pulmonary nodules on chest CT utilizing axial slice number labels.

BACKGROUND: Reidentification of prior nodules for temporal comparison is an important but time-consuming step in lung cancer screening. We develop and evaluate an automated nodule detector that utilizes the axial-slice number of nodules found in radiology reports to generate high precision nodule predictions. METHODS: 888 CTs from Lung Nodule Analysis were used to train a 2-dimensional (2D) object detection neural network. A pipeline of 2D object detection, 3D unsupervised clustering, false positive reduction, and axial-slice numbers were used to generate nodule candidates. 47 CTs from the National Lung Cancer Screening Trial (NLST) were used for model evaluation. RESULTS: Our nodule detector achieved a precision of 0.962 at a recall of 0.573 on the NLST test set for any nodule. When adjusting for unintended nodule predictions, we achieved a precision of 0.931 at a recall 0.561, which corresponds to 0.06 false positives per CT. Error analysis revealed better detection of nodules with soft tissue attenuation compared to ground glass and undeterminable attenuation. Nodule margins, size, location, and patient demographics did not differ between correct and incorrect predictions. CONCLUSIONS: Utilization of axial-slice numbers from radiology reports allowed for development of a lung nodule detector with a low false positive rate compared to prior feature-engineering and machine learning approaches. This high precision nodule detector can reduce time spent on reidentification of prior nodules during lung cancer screening and can rapidly develop new institutional datasets to explore novel applications of computer vision in lung cancer imaging.

CT Features of Benign Intrapulmonary Lymph Nodes in Pediatric Patients With Known Extrapulmonary Solid Malignancy.

OBJECTIVE. The purpose of our study was to determine the CT features of benign intrapulmonary lymph nodes in pediatric patients with known extrapulmonary solid malignancy. MATERIALS AND METHODS. A retrospective review of surgical pathology archives was performed to identify consecutive chest CT studies of pediatric patients (≤ 18 years) with extrapulmonary solid malignancy and histologically confirmed benign intrapulmonary lymph nodes between January 1, 2004, and March 15, 2020. CT features of intrapulmonary lymph nodes-including size, shape, margin, type, associated calcification or fat, and location-were independently evaluated by two pediatric radiologist reviewers. The CT features of benign intrapulmonary lymph nodes in pediatric patients were analyzed using summary statistics. Interobserver agreement was measured with the kappa coefficient. RESULTS. There were 36 pathology-confirmed benign intrapulmonary lymph nodes in 27 pediatric patients (18 boys and nine girls; mean age, 12 years; age range, 1-18.2 years). Twenty-three (63.9%) of the benign intrapulmonary lymph nodes were biopsied from the right lung and 13 (36.1%) from the left lung (p = .03). The mean size, determined from CT studies, of benign intrapulmonary lymph nodes was 3.6 mm (SD, 1.4 mm; range, 1.3-7.8 mm). Triangular shape (25/36, 69.4%) was the most common shape of the benign intrapulmonary lymph nodes. Less commonly seen shapes of benign intrapulmonary lymph nodes were oval (6/36, 16.7%), round (3/36, 8.3%), and trapezoidal (2/36, 5.6%). All benign intrapulmonary lymph nodes were smoothly marginated and solid without associated calcification or fat. Of the 36 benign intrapulmonary lymph nodes, 15 (41.7%) were pleura-based; 11 (30.6%), perifissural; and 10 (27.8%), parenchymal. The kappa value for interobserver agreement between the two reviewers was 0.917 (95% CI, 0.825-1.000; standard error, 0.047), which corresponds to near-perfect agreement. CONCLUSION. In pediatric patients with known extrapulmonary solid malignancy, benign intrapulmonary lymph nodes are subcentimeter (mean size, 3.6 mm), smoothly marginated, and solid without containing calcification or fat on CT. In particular, triangular shape was the most commonly encountered shape of a benign intrapulmonary lymph node.

Domain specific word embeddings for natural language processing in radiology.

BACKGROUND: There has been increasing interest in machine learning based natural language processing (NLP) methods in radiology; however, models have often used word embeddings trained on general web corpora due to lack of a radiology-specific corpus. PURPOSE: We examined the potential of Radiopaedia to serve as a general radiology corpus to produce radiology specific word embeddings that could be used to enhance performance on a NLP task on radiological text. MATERIALS AND METHODS: Embeddings of dimension 50, 100, 200, and 300 were trained on articles collected from Radiopaedia using a GloVe algorithm and evaluated on analogy completion. A shallow neural network using input from either our trained embeddings or pre-trained Wikipedia 2014 + Gigaword 5 (WG) embeddings was used to label the Radiopaedia articles. Labeling performance was evaluated based on exact match accuracy and Hamming loss. The McNemar's test with continuity and the Benjamini-Hochberg correction and a 5×2 cross validation paired two-tailed t-test were used to assess statistical significance. RESULTS: For accuracy in the analogy task, 50-dimensional (50-D) Radiopaedia embeddings outperformed WG embeddings on tumor origin analogies (p < 0.05) and organ adjectives (p < 0.01) whereas WG embeddings tended to outperform on inflammation location and bone vs. muscle analogies (p < 0.01). The two embeddings had comparable performance on other subcategories. In the labeling task, the Radiopaedia-based model outperformed the WG based model at 50, 100, 200, and 300-D for exact match accuracy (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively) and Hamming loss (p < 0.001, p < 0.001, p < 0.01, and p < 0.05, respectively). CONCLUSION: We have developed a set of word embeddings from Radiopaedia and shown that they can preserve relevant medical semantics and augment performance on a radiology NLP task. Our results suggest that the cultivation of a radiology-specific corpus can benefit radiology NLP models in the future.

An Open-Source, Vender Agnostic Hardware and Software Pipeline for Integration of Artificial Intelligence in Radiology Workflow.

Although machine learning (ML) has made significant improvements in radiology, few algorithms have been integrated into clinical radiology workflow. Complex radiology IT environments and Picture Archiving and Communication System (PACS) pose unique challenges in creating a practical ML schema. However, clinical integration and testing are critical to ensuring the safety and accuracy of ML algorithms. This study aims to propose, develop, and demonstrate a simple, efficient, and understandable hardware and software system for integrating ML models into the standard radiology workflow and PACS that can serve as a framework for testing ML algorithms. A Digital Imaging and Communications in Medicine/Graphics Processing Unit (DICOM/GPU) server and software pipeline was established at a metropolitan county hospital intranet to demonstrate clinical integration of ML algorithms in radiology. A clinical ML integration schema, agnostic to the hospital IT system and specific ML models/frameworks, was implemented and tested with a breast density classification algorithm and prospectively evaluated for time delays using 100 digital 2D mammograms. An open-source clinical ML integration schema was successfully implemented and demonstrated. This schema allows for simple uploading of custom ML models. With the proposed setup, the ML pipeline took an average of 26.52 s per second to process a batch of 100 studies. The most significant processing time delays were noted in model load and study stability times. The code is made available at " http://bit.ly/2Z121hX ". We demonstrated the feasibility to deploy and utilize ML models in radiology without disrupting existing radiology workflow.

Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms.

Importance: Mammography screening currently relies on subjective human interpretation. Artificial intelligence (AI) advances could be used to increase mammography screening accuracy by reducing missed cancers and false positives. Objective: To evaluate whether AI can overcome human mammography interpretation limitations with a rigorous, unbiased evaluation of machine learning algorithms. Design, Setting, and Participants: In this diagnostic accuracy study conducted between September 2016 and November 2017, an international, crowdsourced challenge was hosted to foster AI algorithm development focused on interpreting screening mammography. More than 1100 participants comprising 126 teams from 44 countries participated. Analysis began November 18, 2016. Main Outcomes and Measurements: Algorithms used images alone (challenge 1) or combined images, previous examinations (if available), and clinical and demographic risk factor data (challenge 2) and output a score that translated to cancer yes/no within 12 months. Algorithm accuracy for breast cancer detection was evaluated using area under the curve and algorithm specificity compared with radiologists' specificity with radiologists' sensitivity set at 85.9% (United States) and 83.9% (Sweden). An ensemble method aggregating top-performing AI algorithms and radiologists' recall assessment was developed and evaluated. Results: Overall, 144 231 screening mammograms from 85 580 US women (952 cancer positive ≤12 months from screening) were used for algorithm training and validation. A second independent validation cohort included 166 578 examinations from 68 008 Swedish women (780 cancer positive). The top-performing algorithm achieved an area under the curve of 0.858 (United States) and 0.903 (Sweden) and 66.2% (United States) and 81.2% (Sweden) specificity at the radiologists' sensitivity, lower than community-practice radiologists' specificity of 90.5% (United States) and 98.5% (Sweden). Combining top-performing algorithms and US radiologist assessments resulted in a higher area under the curve of 0.942 and achieved a significantly improved specificity (92.0%) at the same sensitivity. Conclusions and Relevance: While no single AI algorithm outperformed radiologists, an ensemble of AI algorithms combined with radiologist assessment in a single-reader screening environment improved overall accuracy. This study underscores the potential of using machine learning methods for enhancing mammography screening interpretation.